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ABSTRACT 

In 1987, a school-university collaborative project. 
Positive Attitudes in Tennessee Schools (PATS) , was established to 
improve school-learning environments. This paper presents findings of 
a study that investigated the effect of school participation in PATS 
on school climate. A secondary focus was to determine which 
school-climate variables could predict differences between 
experienced and beginning PATS schools. Data were derived from the 
administration of the Tennessee School Climate Inventory (TSCi) to 
teachers at 92 Tennessee schools — 51 elementary, 21 middle, and 20 
senior high schools. No significant differences were found to exist 
between experienced and beginning schools. Elementary schools 
demonstrated more instructional improvement than did middle and 
senior high schools, regardless of project experience. TSCI 
dimensions of school climate were poor predictors of program 
experience level. Despite nonsignificant differences between 
experienced and beginner schools, overall net gains were observed for 
all dimensions. Data suggest that 61.5 percent of the schools could 
be expected to improve school-climate variables within 3 to 4 years 
and that only 38.5 percent could be expected to improve such 
variables within one year. A possible explanation for the apparent 
lack of significant improvement is the stability of the 
school-climate construct. (LMI) 
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Abstract 

The relationship between school climate. Positive Attitudes in Tennessee Schools 
(PATS) project participation, and organizational level was investigated. Seven 
dimensions of the Tennessee School Climate Inventory (TSCi) were utilized as 
dependent variables in a 2 X 2 MANCOVA design to assess the effects of PATS 
experience and school level. Additionally, a discriminant function analysis was 
performed to determine those variables which differentiated experienced firom 
beginner schools. No significant effect was observed for the experience by 
organizational level interaction, nor was there a significant effect for program 
experience. There was a significant main effect for organizational level, and the 
TSCI Instruction dimension was significandy influenced by diis effect. Elementary 
schools exhibited greater improvement in the area of Instruction than middle and 
senior high schools. Despite non-significant differences between experienced and 
beginner schools, overall net gains were observed for all dimensions. A derived 
disciminant function correcdy classified 60.9% of schools as either experienced or 
beginner using three TSCI dimensions (Leadership, Environment, and Instruction), 
though this function was not statistically significant. The data suggest that 61.5% 
of schools would be expected to improve school cUmate variables over a three- to 
four-year period, while only 38.5% of schools would be expected to improve over 
a one-year period The stability of the school climate construct was implicated as a 
possible explanation for the apparent lack of significant improvement as a function 
of program participation. 
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The Relationship Between School Qimate, 
PATS Program Participation, and Organizational Level 

In 1989 a school and university collaboration known as Positive Attitudes m 
Tennessee Schools (PATS; Pike & Chandler, 1989) was established to improve 
school learning environments. The collaboration involved school leadership teams, 
state department of education personnel, and university' faculty cooperatively 
planning and implementing a school reform initiative designed for a five-year 
period The program was initiated when the first training academy was held in the 
summer of 1989 for leadership teams representing 41 schools selected as Pilot 
sites. Initial learning environment audits were conducted in the fall semester of 
1989. Subsequent audits were conducted in the Pilot sites in the spring semesters 
of 19901993. Additional schools became involved in the project in the following 
years: Cycle II (1990-1991), and Cycle III and Extended (1992-1993) sites. 
Overall, 169 schools have participated during die four years of die project, 
representing more than 1 1,000 faculty and staff respondents. 

Conceptual Frameworic for the School Improvement Project 

The use of data-based, or informed, decision making and planning as a 
sUntegy in school improvement initiatives emerged in recent years as a strategy for 
improving the performance of America's public schools (Ross & Mahlck, 1990; 
Wilson, Miller, & Rossman, 1985). The indicator system developed for PATS 
was grounded on tlie belief that school contextual information could be used 
effectively by school personnel to plan and implement school improvement 
activities that could enhance the quality of woriq)lace environments for teaching and 
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learning environments for students (Butler & Alberg, 1993; David, 1987; Ross & 
Malck, 1990). 

This research focused on the school as a unit. From a policy-making 
perspective, the school as a unit of change has become politically important in 
recent years (Heck & Mayor, 1993). Reform efforts such as school choice, school- 
based decision making, usr. of standards and accountability, along with efforts to 
study the characteristics of effective schools, are grounded on the belief that schools 
as organizational units can be altered and evaluated in terms of their effectiveness or 
productivity. Development of systems for profiling school contextual indicators is 
associated with this current policy perspective (Oakes, 1986, 1989). Strategies for 
generating contextual indicators as organizational attributes as they exist over time 
have been advocated by educational many reformers, researchers, and political 
leadei-s (Heck & Mayor, 1993; Porter, 1991; Rumbcrger & Willms, 1992). 

One of the first steps involved in developing a system of collecting and 
reporting school context indicators necessitated a conceptual formulation of what is 
meant by a school's learning environment. In PATS, a learning environment is 
defined as a composite of school and classroom sociopsychological factors which 
influence student achievement, attitudes, and conceptions of self (Stockard & 
Mayberry, 1985). Data utilized as indicators of the quality of school organizational 
contexts were perceptual information from school professional personnel and 
students. Professional personnel were viewed as appropriate sources of information 
through which overall school climate or culture dimensions were constructed. 
Students were expected to provide class context data and perceptions of themselves 
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as learners. Three different inventories were developed to generate these measures. 

Numerous organizational, social, cultural, personal, and physical factors 
influence school contexts (Anderson, 1982; Deal, 1993; Heck & Mayor, 1993; 
Purkey & Smith, 1983), yet many valued characteristics are often beyond current 
measurement technology (Oakes, 1989). The system of indicators used to generate 
data used in this report focused primarily on cultural aspects of schools: beliefs, 
values, relationships, and expectations believed to affect the quality of both teaching 
and learning (Deal, 1993). 

The nature of the project also involved specifying a change strategy 
appropriate for accomplishing the goals of the school improvement initiative. 
Dissemination was selected as the change model to guide planning and 
implementation focusing on data-based school improvement As a policy tool, the 
dissemination model relies heavily on the use of knowledge-based products by 
external agents in stimulating and supporting planned change at the school level 
(Firestone & Corbett, 1988). 

Specific change features were based upon several key considerations: 

1. Program products (instructional materials, data summaries) 

were to be non-threatening. 

2. Leadership teams were to be trained to introduce the materials and activities to 

school faculties and coordinate project implementation in each site. 

3. Overall management of the program was the resp onsibiiiiy of state department 

personnel who would provide technical assistance, coordinated training 
academies and workshops, and facilitate communication between the sites, 

4. A host of school and community factors that both support and inhibit educational 

change were acknowledged. 

5. Program implementation and results were linked to individual school leadership, 

resources, and press for change. 

6. Innovative practices within the sites were anticipated as outcomes of the 

implementation process. 

7. Successful results in the sites were to be disseminated to other sites through 

annual academies, regional workshops, and other communication strategies. 
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The change model utilized incorporates the recog.udon that organizarioi-al 
and environmental changes may depend upon modifications of the normative 
structures of schools that support or inhibit change and improvement. Thus, 
information collection and reporting strategies were designed to describe the 
normative structures of the participating schools at selected times and to explain, if 
possible, major changes and influential factors responsible for improvements 
realized 

An analysis of the 1989 baseline data was conducted and a summary 
prepared of empirical indicators reflecting the normative structures which directed 
teaching and learning in the Pilot schools during die fall semester of 1989 (Butler, 
1990). Generalizations reported were based on aggregated state- wide data using 
elementary, junior high/middle schools, and senior highs schools as units for 
analysis. Various organizational context strengths and weaknesses reflected in the 

empirical indicators were noted The report concluded with the following: 

Whether the realities of school learning environments measured during the 
fall of 1989 will be similar at later points in time is unknown. Data generated in 
the Spring of 1990, and in subsequent years, will be utilized in answering 
questions relative to changes and stability in patterns over time. School 
improvement plans being developed and implemented to produce changes in 
learning environments may indeed yield outcomes which differ bom those 
observed in the baseline data. The organizational and behavioral patterns 
reflected in these data, however, may have consistency and stability difficult to 
modify lluder, 1990, p. x). 

The present study investigated the relationship between school climate, 

school organizational level, and PATS program participation. The intent was to 

determine if school climate dimensions remained stable in light of conceited efforts 

to improve them. In addition, the ability of school climate variables to discriminate 

between experienced and beginner PATS schools was investigated. 



School Climate 
7 

Method 

School Sample 

The ninety-two schools providing data for this research were those that had 
participated in a school climate audit during the Spring of 1993 and that had 
completed the designated baseline audit for a specific cohort Characteristics of 
these schools are presented in Table 1. The 51 elementary schools represented 
55.4% of the sample. The 21 middle and 20 senior high schools comprised 22.8% 
and 21.7%, respectively. Most schools were located in the eastern part of the state, 
though the middle and west regions were adequately represented More than two- 
tliirds of the schools served a rural or small town population; about one-third were 
urban or suburban. The number of teachers in each school ranged from 8 to 80, 
with elementary schools having the fewest and senior high schools having the most 
teachers on average. This school sample represents more thar 6,000 respondents. 

The Tennessee School Climate Inventory (TSCI; Buder & Alberg, 1991) is 
composed of seven scales, or dimensions, which are based on factors logically 
constructed from variables believed to be associated with effective schools and 
organizational climates (Anderson, 1982; Purkey & Smith, 1983; Walberg, 1987). 
Data from the TSCI and other instruments, collectively comprising the Learning 
Environment Assessment System (Butler & Alberg, 1990a), have been used to 
plan school improvement initiatives. The seven dimensions are: Order, 
Leadership, Environment, Involvement, Instruction, Expectations, and 
Collaboration. The inventory consists of 49 items (seven per dimension) which are 
rated using a five-point scale (l=strong agreement, 5=strong disagreement). 
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Table 1 

Characteristics of the School Sartiple 



School Level 



Variable 




Elementary 
(r. = 51) 


Middle 
(« = 21) 


Senior High 
(n = 20) 


Total 
(N = 92) 


% 


State Region 


East 


21 


9 


8 


38 


41.3 




Middle 


17 


7 


4 


28 


30.4 




West 


13 


5 


8 


26 


28.3 


Coimnunity 


Rural 


19 


6 


7 


32 


34.8 




Small Town 18 


7 


6 


31 


33.7 




Urban 


11 


6 


6 


23 


25.0 




Suburban 


3 


2 


1 


6 


6.5 


Number of 
Teachers 


Mean 


23.8 


33.9 


42,9 


30.3 






Range 


8-55 


19-49 


10-80 


8-80 
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The range of scores is 7-35, with higher scores associated with more positive 
learning environments. 

The concept of school climate used to develop the TSCI encompasses 
"norms, beliefs^ and attittides reflected in institutional patterns and practices that 
enhance or impede student achievement" (Wallich, 1981). The notion of climate is 
similar to that of "culture" which includes belief systems, values, general cognitive 
structures, and meanings that govern patterned relationships of persons and groups 
(Tagiuri, 1968). 

Internal consistency alpha coefficient estimates for the entire instrument are 
.96 at the elementary school level, .97 at the middle school level, and .93 at the 
senior high school level; alpha coefficients for the seven school climate scales 
range from .67 to .96 (Butler & Alberg, 1990b). Test-retest reliability estimates for 
the dimensions, based on school-wide data after a four month interval, range from 
.61 to .78 (Kenney, 1993a). 
Design and Procedure 

A 2 X 2 multivaiiate analysis of covariance (MANCOVA) was employed 
using the seven TSCl dimensions as dependent variables to assess the impact of 
program participation and school level. A discriminant function analysis assessed 
the predictive capability of TSCI dimensions in differentiating experienced from 
beginner schools using current performance (spring 1993 data). 

Schools were categorized as experienced if they had participated in at least 
four school climate audits. That is, they had participated in a baseline and at least 
three subsequent audits (three or more years of program activity). The beginner 
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schools constituted those institutions that had participated in two audits (i.e., only 
one year of actual program activity). To obtain relatively equal numbers of schools 
in each cell of the 2 X 2 design, middle and senior high schools were grouped 
together into a single category of school level which was then compared to an 
elementary category. Such an approach is statistically sound since elementary 
schools have been found to differ significantly from middle and senior high schools 
on all TSCl dimensions (Kenney, 1993b). 

The TSCI scores used for this research were in the form of T-scores (mean 
- 50, standard deviation = 10). These standardized scores are linear 
transformations based on all baseline scores of 129 elementary, middle, and senior 
high schools, represez*}i :\g more ii -Ji 4,300 respondents (Kenney, 1993b). 

Results 

MANCQVA 

No interaction between experience and organizational level was detected, 
Wilks A = .93, F(7,75) = .77, p = .618. There was also no main effect for 
experience, Wilks A = .88, F(7,75) = 1.50, p = .179, 

A main effect for school organizational level was found, Wilks A = .75, 
F(7,75) = 3.59, p = .002. Inspection of univariate F-ratios indicated that only the 
Instruction dimension was significantly influenced by this main effect, F(l,81) = 
5.25, p = .025 (see Table 2). Elementary schools had larger increases in the 
perceived efficacy of instructional methods than did middle and senior high schools 
(see Figure 1). All changes in mean T-scores were ordinal. That is. Experienced 
schools had higher scores than Beginner schools on all dimensions (see Figure 2). 
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Table 2 

School Climate T-scores by Program Experience and Scfiool Level 



Beginner Experienced Total 



School Level Base 1993 A Base 1993 A Base 1993 A 



Elementary 



Older 


50.4 


Leadership 


50.5 


Environment 


49.4 


Involvement 


50.2 


Instruction 


51.4 


Expectations 


50.7 


Collaboration 


49.4 



Middle/Senior High 

Oder 53.4 

Leadershq) 52.0 

Environment 52.4 

Involvement 51.2 

Instruction 53.5 

Expectations 52.7 

Collaboration 52.8 

Total 

Oder 51.6 

Leadership Cl.l 

Environment 50.6 

Involvement 50.6 

Instruction 52.2 

Expectations 51.5 

Collaboration 50.7 





n o 
-u.y 


J i.y 




50.7 


0.2 


51.0 


53.3 


50.4 


1.0 


52.8 


53.8 


52.1 


1.9 


50.4 


54.3 


55.0 


4.6 


50.5 


61.3 


51.4 


0.7 


51.8 


56.7 


51.6 


1.2 


52.4 


55.3 


53.0 


-0.4 


50.6 


51.7 


52.8 


0.8 


51.3 


51.0 


53.8 


1.4 


52.7 


55.0 


53.6 


2.4 


50.9 


53.5 


55.2 


1.7 


51.3 


55.3 


53.4 


0.7 


51.4 


52.9 


55.3 


2.5 


51.9 


54.8 


50.9 


-0.7 


51.3 


53.4 


51.5 


0.4 


51.2 


52.2 


51.7 


1.1 


52.8 


54.4 


52.7 


2.1 


50.7 


53.9 


55.1 


2.9 


50.9 


58.3 


52.2 


0.7 


51.6 


54.8 


53.1 


2.4 


52.2 


55.1 



t 1 


^1 1 


jZ.j 


1 O 

l.Z 


2.3 


50.8 


52.0 


1.2 


1.0 


51.1 


52.1 


1.0 


3.9 


50.3 


53.2 


2.9 


10.8 


51.0 


58.2 


7.2 


4.9 


51.3 


53.5 


2.6 


2.9 


50.9 


53.5 


2.6 


1.1 


51.6 


52.2 


0.6 


-0.3 


51.6 


51.7 


0.1 


2.3 


52.6 


54.5 


1.9 


2.6 


51.0 


53.5 


2.5 


4.0 


52.2 


55.3 


3.1 


1.5 


51.9 


53.2 


1.3 


2.9 


52.3 


55.0 


2.7 


2.1 


51.4 


52.2 


0.8 


1.0 


51.1 


51.8 


0.7 


1.6 


51.8 


53.1 


1.3 


3.2 


50.6 


53.4 


2.8 


7.4 


51.5 


56.8 


5.3 


3.2 


51.6 


53.6 


2.0 


2.9 


51.5 


54.2 


2.7 



Note. A = change, Elemenury Beginner, n = 26; Elementary Experienced, n = 25; Middle/Senior High 
Beginner, n = 17; Middle/Senior High Experienced, n = 24. 



12 



School Climate 
12 



Figure 1 

School Climate Instruction Dimension by School Level 




Baseline 



Audit 



1993 



# Elementary 

O — Middle/Senior High 
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Figure 2 

Changes in T-scores for Beginner and Experienced PATS Schools 
10 1 r 
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Disciminant Analysis 

The results of the discrimiant analysis are detailed in Table 3. Three 
dimensions were selected using Wilks' method, though as a combination these 
variables did not significantly discriminate Experienced from Beginner schools, 
Wilks A = ,95, F(l,3) = 1.65, p = , 183, The derived discriminant function 
coirectly classified 60.9% of schools. The resultant canonical correlation between 
three selected dimensions (Leadership, Environment, and Instruction) and 
experience level (0 = Beginner, 1 = Experienced) was .23. 

EHscussion 

This research investigated the relationship between school climate, PATS 
program experience, and school organizational level. No significant differences 
were found between experienced and beginner schools, nor was any interaction 
effect found for experience level and school level. School level was found to have 
a significant effect, with elementary schools improving more than middle and senior 
high schools, regardless of experience (i.e., time) in the program. In addition, 
current (spring 1993) TSCI dimensions were found to be relatively poor predictors 
of program experience level among the schools. 

Several explanations may be proposed in light of tiiese results. First, it may 
be concluded that the PATS program has no effect on improving school climate. 
On the surface, this conclusion seems reasonable smce schools did not significandy 
improve scores over tirae in relation to particular starting points. Examination of 
the changes in scores, however, indicates that, with the exception of Order in the 
case of Beginner schools, all other dimensions showed net improvement over time 
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Table 3 

Discriminant Function Coefficients of Three School Climate Dimensions for the 
Prediction of Experience Level 



Step 


Dimension 


Unstandardized 
Coefficient 


Standardized 
Coefficient 


Wilks' 

A 


P 


1 


Leadership 


-0.145 


-1.52 


.977 


.151 


2 


Environment 


0.118 


1.18 


.962 


.178 


3 


Instruction 


0.078 


0.85 


.947 


.183 




(Constant) 


-3.150 









Note, Coefficients axe based on Beginner = 0, Experienced 1. 
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even among beginner schools. The data suggest that 61,5% of schools would be 
expected to improve school climate variables over a three- to four-year period, 
while only 38.5% of schools would be expected to improve over a one-year period. 
This fact would appear to provide some evidence for the positive impact of the 
PATS project on school climate dimensions. 

A second explanation for the results may be the stability of school climates 
themselves. A review of the present data revealed moderate to high test-retest 
correlations, ranging from .43 to .83. Regardless of experience level, test-retest 
relationships remained stable, though elementary coefficients were somewhat 
higher than middle school and senior high coefficients (see Table 4). 

This research suggests that changing school climate may be a long-term 
process. Schools with three to four years of experience trying to improve their 
respective school environments were not found to have improved (at least 
perceptually) any more than schools with one year of experience. However, 
experienced schools were more likely to have shown net improvement over time 
than beginner schools. The stability of school climates may be contributing to the 
lack of significant improvements. Future research in this area should focus on the 
resiliency of school climate and its effects on the process of reform. 

While school cultures or climates seem to be inherently stable and resistant 
to change, the PATS project has demonstrated that they are subject to modification 
and improvement PATS has also verified that successful strategies can be 
developed to structure and communicate information useful in highlighting and 
defining problem areas or issues in schools, not in designing remedies. These 



17 



School Climate 
17 



Table 4 . 

Test-Retest Coefficients by Experience Level and School Level 

Beginner Experienced 
School Level Range (Median) Range (Median) 



Elementary 

Middle/ 
Senior High 

Total 



.63 to .78 (.72) 

.43 to .83 (.50) 
.43 to .83 (.64) 



.57 to .80 (.70) 

.45 to .78 (.61) 
.45 to .80 (.64) 



Total 
Range (Median) 
.57 to .80 (.71) 

.43 to .83 (.60) 
.43 to .83 (.64) 



Note. Elementary Beginner, n = 26; Elementary Experienced, n = 25; Middle/Senior High 
Beginner, n = 17; Middle/Senior High Experienced, n = 24, 
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results suggest that Tennessee is achieving success in addressing the difficulty of 
connecting information to inform practitioners in ways that will assist and motivate 
them to change their perspectives and practices. Moreover, the educational 
information system implemented in PATS demonstrates that measures of what goes 
on in schools is important information that can be used in discussions about how to 
improve schools as organizational entities. Other studies are currently being 
planned to determine if changes in school contexts and indicator systems produce 
changes in outcomes over time. 
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